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SUMMARY 


Most  attempts  to  apply  statistical  theory  to  prediction  of 
"new"  materiel  characteristics  have,  by  a strange  rationale,  neglected 
the  design  process  by  which  they  are  determined.  The  analyst  or 
statistician  and  the  engineer  or  designer  generally  do  not  understand 
the  interrelation  of  their  respective  disciplines  and  each  has  poor 
understanding  of  the  basic  concepts  in  the  field  of  the  other.  They 
fail  to  recognize  that  the  "cause  and  effect"  relationships  basic  to 
most  design  techniques  are  established  on  sound  statistical  principles. 

The  statistician  tends  to  distrust  the  "deterministic"  relationships 
used  by  the  designer,  who  regards  statistics  as  a means  of  evaluating 
tolerances  and  errors.  In  particular,  the  statistician  does  not  realize 
that  design  processes  constitute  the  "ideal"  determinants  of  the  content 
of  populations  to  be  used  in  predicting  characteristics  of  the  products 
of  those  processes.  Predicting  has  been  attempted,  based  on  factors 
that  are  essentially  unrelated  to  the  true  differences  between  the  "old" 
items,  which  have  provided  the  experience  history  or  data,  and  the  new 
or  future  items.  Regardless  of  any  opinions  that  might  be  entertained 
about  them,  design  processes  are  the  determinants  of  those  differences, 
and  the  success  of  any  prediction  technique  will  depend  upon  how  well 
it  is  correlated  with  these  processes.  Communication  between  designers 
and  analysts  has  recently  been  promoted  and  significant  progress  is 
claimed.  There  is  wide  recognition  of  the  value  of  anticipating  problems 
in  areas  such  as  reliability  before  an  item  is  produced,  but  as  yet 
there  is  little  evidence  that  the  crux  of  the  prediction  technique  has 
been  generally  recognized.  While  the  missing  ingredients  are  the  design 
processes,  these  serve  to  define  populations;  therefore  the  statistician 
must  accept  the  responsibility  for  asking  the  right  questions  and  using 
the  answers  properly.  As  one  writer  puts  it,  "The  statistician  cannot 
excuse  himself  from  the  duty  of  getting  his  head  clear  on  the  principles 
of  scientific  inference — ."  Because  little  historical  data  includes 
information  about  true  physical  phenomena,  it  will  be  necessary  to 
generate  new  data  bases  to  support  effective  prediction.  In  this  effort 
statisticians  will  also  need  information  from  designers.  A fruitful 
source  is  design  testing,  the  results  of  which  have  often  been  stigmatized 
as  "laboratory  data."  If  an  effective  rapport  between  statisticians 
and  designers  can  be  achieved,  the  resulting  improvement  in  prediction 
capability  will  enhance  the  value  of  activities  such  as  Maintenance 
Engineering  Analysis,  Reliability  Growth  Analysis,  Risk  Analysis,  and 
Value  Engineering. 
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RSM  PREDICTION  IN  ENGINEERING  DEVELOPMENT 


1.  INTRODUCTION 

Certain  applications  of  statistical  (and  probability)  theory 
are  confused  by  an  assumed  but  false  dichotomy  between  the  disciplines 
concerned  with  statistics  and  those  concerned  with  the  phenomena  involved 
in  its  application.  This  is  particularly  true  in  evaluation  of  the 
development  of  "new"  systems.  The  generally  accepted  reference  books 
on  statistics  describe  these  disciplinary  interrelationships,  and  in  the 
more  traditonal  applications  they  are  appropriately  considered.  Their 
proper  use  in  statistical  prediction  will  be  defined  as  a prelude  to  a 
discussion  of  methods  and  techniques. 

1. 1 Statistics  and  Science. 

"Statistics  is  a branch  of  scientific  method  which  deals  with 
the  data  obtained  by  counting  or  measuring  the  properties  of  populations 
of  natural  phenomena.  In  this  definition  'natural  phenomena'  includes 
all  the  happenings  of  the  external  world  whether  human  or  not." 

(Reference  1,  p.  2).  Thus  statistics  "counts  or  measures  the  properties" 
of  that  which  science  "explains"  in  terms  of  cause  and  effect.  The 
introduction  of  most  reference  books  essentially  agree  with  this  definition 
(References  2 through  7,  see  page  23  herein).  Reference  2 (pp  1 § 2) 
addresses  the  interrelationship  of  science  and  statistics  in  the  context 
of  the  function  and  scope  of  the  respective  disciplines,  showing  that 
statistics  provides  a validation  of  propositions  or  hypotheses  that  are 
induced  by  science.  "Statistics"  as  used  herein  includes  the  probability 
theory  applicable  to  analyzing  the  effects  of  the  characteristics  of  a 
system  on  its  use  and  support.  Those  who  are  responsible  for  this 
analysis  ("reliability  engineers",  "maintainability  engineers",  logisti- 
cians", etc.)  will  be  called  "statisticians"  or  "analysts"  for  brevity. 
Those  engaged  in  engineering  design  and  development  will  be  called 
"designers" . 

1.2  The  Scientific  Method. 


Science  is  assumed  to  mean  the  development  of  systematic 
knowledge  by  inductive  reasoning,  based  on  observations  of  natural 


^Kendall,  M.  G.,  and  Stuart,  A.,  The  Advanced  Theory  of  Statistics, 
Vol  1,  3rd  Ed.  (1969),  Hafner,  New  York. 

2 

Dixon,  W.  J.,  and  Massey,  F.  J.,  Jr.,  Introduction  to  Statistical 
Analysis , (1951),  McGraw-Hill,  New  York. 
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phenomena.  The  following  are  quoted  from  Reference  3:  "I  have  assumed, 

as  the  experimenter  always  does  assume,  that  it  _i£  possible  to  draw  valid 
inferences  from  the  results  of  experimentation:  that  it  is  possible  to 

argue  from  consequences  to  causes,  from  observations  to  hypotheses;" 

(p.  3);  "Inductive  inference  is  the  only  process  known  to  us  by  which 
essentially  new  knowledge  comes  into  the  world"  (p.  7);  and,  "The 
statistician  cannot  excuse  himself  from  the  duty  of  getting  his  head 
clear  on  the  principles  of  scientific  inference — " (p.  2). 

The  "scientists"  with  whom  we  are  presently  concerned  are  not 
developing  "new  knowledge"  or  basic  principles  (research).  Nevertheless, 
the  development  of  anything  that  differs  in  any  way  from  direct  previous 
experience  includes  the  basic  inductive  process. 

1. 3 Application  of  Statistics. 

In  most  applications  of  statistics,  assumptions  requiring 
attention  involve  distributions,  randomness,  etc.  Identification  of  the 
characteristics  of  prospective  elements  that  qualify  them  as  belonging 
to  a particular  population  is  usually  obvious  and  special  expertise  is 
not  required.  For  prediction,  however,  this  is  not  true.  Since  the 
"new"  system  differs  from  those  on  which  history  is  available,  it  is 
apparent  that  the  validity  of  the  prediction  depends  on  the  valid  identi- 
fication of  these  differences  and  their  proper  incorporation  in  the 
statistical  analysis.  The  differences  are  established  by  the  designer 
(to  meet  functional  objectives).  But  the  statistician  identifies  these 
differences  primarily  on  the  basis  of  names  of  system  units  and  on 
correlations.  System  units  (e.g.,  electronic  "black  boxes")  vary 
drastically  in  content  and  complexity  and  are  given  functional  names. 

The  hazards  of  inferences  drawn  entirely  from  correlation  are  illustrated 
in  Reference  4,  page  52,  in  discussion  of  the  "logistic  distribution 
function".  Neither  method  has  a direct  relation  to  the  causes  of 
failures,  that  is  to  stresses  exceeding  strengths. 

1.4  Application  of  Prediction. 

The  Appendix  describes  an  Engineering  Reliability  Program  from 
the  point  of  view  of  a contractor  making  a proposal  to  the  Department  of 
Defense.  The  first  three  sections  were  prepared  as  a guide  after  a study 
of  a particular  contractor's  experience  on  several  programs  and  a review 
of  military  standards  and  requirements  (circa  1967) . Section  Four  is  a 
brief  review  of  the  most  significant  result  of  a recently  completed 


3 

Fisher,  R.  A.,  The  Design  of  Experiments,  8th  Ed.  (1950),  Hafner,  New  York. 

4 

Feller,  Win. , An  Introduction  to  Probability  Theory  and  its  Application, 

Vol  2,  (1966),  John  Wiley,  New  York. 
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contract  from  AMSAA  to  the  University  of  Delaware  (Contract  No.  DAAD-70- 
C-0011)  entitled  "Research  Study  of  Mechanical  and  Structural  Reliability 
Prediction."  (Reference  8)  The  experience  of  those  performing  the  study 
included  the  field  of  Operations  Research,  and  the  principal  application 
difficulty  in  this  area  and  in  the  area  of  the  Reliability  Prediction 
was  found  to  be  similar  - namely,  the  valid  identification  of  the 
phenomena  involved  in  the  application,  and  the  correlation  of  the  analyti- 
cal theory  and  techniques  with  them.  The  problem  involves  specialization 
as  well  as  theory  and  practice.  Not  that  there  are  only  "theorists"  on 
one  hand  and  only  "practitioners"  on  the  other.  The  specialists  in 
analysis  and  the  specialists  in  the  phenomena  being  analyzed  generally 
have  a good  grasp  of  both  the  theoretical  and  practical  factors,  each  in 
his  own  field.  But  each  has  a poor  concept  of  the  practical  aspects 
of  the  other  field,  while  viewing  the  ideas  of  specialists  in  that  field 
as  theoretical.  A typical  judgement  by  a designer  is  represented  by 
the  expression  "You  can  prove  anything  with  statistics."  This  is 
matched  by  typical  judgements  of  statisticians,  viewing  the  design  as 
mainly  trial  and  error,  whereby  the  product  is  first  fabricated  with  a 
"reliability"  (or  "MTBF")  of  about  10  percent  of  that  required;  the  final 
product  then  being  "developed"  by  a sequence  of  statistical  analyses  of 
test  data,  more  trial  and  error,  etc.,  all  dignified  by  the  expression 
"corrective  action".  The  evidence  used  for  this  is  failure  data  which 
does  include  the  effects  of  the  normal  design  process,  but  also  includes 
other  effects,  some  of  which  are  not  recognized.  This  will  be  discussed 
further  in  Section  5.2. 

1 . 5 The  Context  of  Prediction. 

The  difficulties  of  making  useful  and  effective  predictions 
during  system  design  are  being  examined.  A number  of  factors  are  involved 
and  they  are  so  interrelated  that  it  is  difficult  to  appreciate  the  full 
significance  of  any  one  without  some  understanding  of  the  others.  As 
these  factors  are  described,  reference  will  be  made  to  their  relevance, 
but  this  may  not  be  too  clear  until  viewed  jointly  and  in  perspective  in 
Section  3.3.  Prediction  will  be  presented  as  it  applies  to  Engineering 
Reliability,  a familiar  area  that  affects  both  use  and  support  of  equip- 
ment. The  basic  principles  apply,  however,  to  other  areas,  such  as 
Maintenance  Engineering,  Reliability  Growth,  and  Risk  Analysis. 


2.  ENGINEERING  RELIABILITY 
2. 1 History. 

During  the  1950's  the  Department  of  Defense  recognized  the  need 
for  a special  effort  during  the  design  and  development  of  materiel,  in 


g 

Stark,  R.  M.  and  Yang,  C.  Y.,  Research  Study  of  Reliability  Prediction 
of  Mechanical  and  Structural  Systems,  Final  Technical  Report,  US  Army 
Contract  No.  DAAD-70-C-0011) , University  of  Delaware. 
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order  to  achieve  adequate  reliability.  This  resulted  in  establishment 
of  the  Advisory  Group  on  Reliability  of  Electronic  Equipment  (AGREE) 
which  issued  a report  (Reference  9)  in  1957  usually  called  the  AGREE 
Report.  By  1960  major  system  contractors  were  required  to  organize  a 
specific  department  in  engineering  to  improve  reliability.  This  was  the 
formal  beginning  of  Engineering  Reliability  . It  was  often  stated  that 
reliability  should  be  "designed  into"  rather  than  "tested  into"  a 
product,  the  latter  referring  to  redesign  after  failure  during  test. 

The  AGREE  study  provided  a background  for  MIL-HDBK-217A  (Reference  10), 
"Reliability  Stress  and  Failure  Rate  Data  for  Electronic  Equipment," 

2. 2 New  Systems. 

It  is  important  to  differentiate  two  Engineering  Reliability 
activities.  One  provides  an  input  to  corrective  action  when  failures 
occur  in  prototype  and  initial  production  tests.  The  expertise  for  this 
derives  mainly  from  the  relatively  mature  disciplines  of  quality  control, 
supplemented  by  close  liaison  with  design.  The  other  is  intended  to 
support  the  design  effort  through  analysis  and  reliability  prediction. 

The  primary  purpose  of  the  latter  is  to  anticipate  and  avoid  reliability 
problems  created  by  new  designs.  This  represents  the  greatest  potential 
benefit  from  Engineering  Reliability.  A second  purpose  is  to  provide 
data  for  trade-off  decisions  during  development  and  preliminary  design. 

What  is  the  essence  of  the  "new"  characteristics  of  new 
systems?  The  answer  must  depend  on  how  they  are  determined.  The 
specific  activity  that  determines  them  is  design,  mainly  in  response 
to  functional  requirements.  Regardless  of  how  requirements  are 
established  and  constraints  imposed,  all  characteristics,  including  any 
that  might  be  "for  reliability  only",  are  achieved  through  incorporation 
in  the  design  of  the  product.  This  is  true  although  understanding  of 
the  phenomena  involved  is  always  limited  by  the  "state-of-the-art" 
and  techniques  may  be  partly  empirical. 

2. 3 Failure  Classification. 

Failure  is  defined  by  MIL-STD-721B  as  "The  inability  of  an  item 
to  perform  within  previously  specified  limits."  Criticisms  of  this  defini- 
tion have  generally  failed  to  recognize  that  it  is  "hardware"  oriented  to 
establish  criteria  for  acceptance  tests.  Limited  definitions  are  essential 
for  most  uses,  but  care  should  be  used  in  formulating  the  limitations  in 
light  of  ultimate  use  of  the  item,  when  all  failures  are  significant. 


9 

Government  Printing  Office,  Reliability  of  Military  Electronic 
Equipment,  4 June  1957. 

^^MIL-HDBK-217  was  issued  on  8 August  1962  and  superseded  by  MIL-HDBK- 
217A  on  1 December  1965. 
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But  are  failures  in  ultimate  use  always  obvious?  Suppose  the  system  was 
not  designed  to  perform  under  the  conditions  in  which  it  was  used,  should 
unsuccessful  results  then  be  classed  as  a failure?  The  answer  is  not 
obvious.  Information  about  the  conditions  may  not  have  been  available 
before  use  of  the  system  was  attempted.  A mission  may  be  started  when 
it  is  known  that  circumstances  may  develop  that  will  preclude  a successful 
outcome.  If  this  happens  the  mission  has  failed,  but  not  the  system  or 
its  components.  Thus  it  is  essential  to  identify  what  has  failed  and 
this  is  not  always  as  clear  as  in  the  example.  We  are  concerned  here 
mainly  with  equipment  failures  and  their  relation  to  design.  Several 
real  or  apparent  anomalies  will  be  considered: 

(1)  A statistically  specified  stress  requires  a fixed  value 

to  be  chosen  as  a criterion  for  acceptance  tests.  A failure  from  exceeding 
this  value  is  or  course  significant  for  ultimate  use,  but  is  not  classed 
as  an  equipment  failure.  Note  that  in  a strict  sense  all  stresses  are 
statistical,  but  when  the  variance  is  very  small  they  are  assumed  constant. 

(2)  If  the  skill  of  an  operator  is  a factor  in  success,  then  the 
stress  is  related  (inversely)  to  skill  and  the  strength  is  the  "opera- 
bility" of  the  equipment.  These  are  established  experimentally,  using 
adequate  samples  of  operators  and  equipments.  Sometimes  the  results 

can  be  expressed  only  in  terms  of  whether  stress  exceeds  strength,  but 
usually  strength  can  be  related  to  measurable  properties,  thus  establish- 
ing specifications  to  identify  equipment  failures. 

(3)  During  development  the  relationship  between  a functional 

requirement  and  characteristics  subject  to  known  design  techniques  may 
be  uncertain,  i.e.,  not  established  by  experience.  In  such  cases  the 
design  is  based  on  an  a priori  conclusion  that  it  will  perform  the  func- 
tion. When  the  design  is  tested,  usually  in  a "breadboard"  or  "prototype" 
form,  it  may  fail  in  two  ways:  the  intended  design  characteristics  may 

not  be  achieved,  a design  failure;  or  the  design  characteristics  may  not 
enable  the  intended  function  to  be  performed,  a failure  of  the  analysis 
leading  to  the  a priori  conclusion.  Both  failures  are  in  a different 
category  from  others  being  considered,  since  they  do  not  evaluate  or 

even  relate  to  the  ultimate  form  of  the  equipment  to  be  produced. 

Instead  they  relate  to  the  result  of  the  design  process,  and  if  they 
occur  there  is  a redesign.  While  not  useful  for  reliability  prediction, 
data  on  redesign  is  useful  for  prediction  of  the  development  process 
itself.  This  application  will  be  considered  in  Section  5. 

(4)  When  the  available  information  about  failures  is  not 
sufficient  to  relate  them  to  known  or  determinable  strengths,  they 
are  usually  called  random.  This  term  does  not  necessarily  imply  a 
specific  statistical  significance,  but  rather  that  a "random"  or 
unidentified  phenomenon  is  involved.  When  random  failures  are  a 
part  of  the  history  on  which  reliability  predictions  are  based,  they 
should  be  properly  included  in  the  analysis. 
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(5)  Typically  when  a stress  exceeds  a strength  the  result  is 
damage,  but  not  necessarily.  Stresses  may  render  an  item  inoperative, 
but  with  no  lasting  effect.  If  the  stresses  are  within  previously 
specified  limits  an  equipment  failure  is  identified. 

(6)  A failure  may  be  due  to  known  and  expected  statistical 
deviations  in  parts.  Or  it  may  be  a "debugging"  failure  known  to  have 
a neglible  chance  of  affecting  the  delivered  equipment.  These  failures 
may  occur  in  all  examples  discussed  above  and  their  categorization  and 
significance  are  routine  and  obvious. 

Failures  have  also  been  classified  as  to  mode,  primarily  in 
terms  of  symptom,  function,  or  effect.  The  pertinent  and  critical 
aspects  of  failure  for  reliability  prediction  have  thus  been  neglected. 

For  example,  a mode  of  failure  of  a mechanism  might  be  labeled  "bearing 
seizure,  obviously  an  effect.  If  the  modal  classification  also  included 
a cause,  such  as  lubrication,  surface  finish,  alignment,  etc.,  this  would 
relate  the  failure  to  those  design  processes  which  determine  resistance  to 
stresses,  the  means  of  "implementing"  reliability  improvements. 

2.4  Failure  Modes  of  a System. 

The  ultimate  concern  of  Engineering  Reliability  is  usually 
with  some  form  of  aggregation  called  a "system".  Prior  to  adequate 
tests  on  the  complete  system,  determining  its  reliability  requires  that 
it  be  analyzed  or  dissected  into  appropriate  elements,  for  which 
reliability  indices  are  derived,  and  then  synthesized  to  yield  system 
reliability.  The  appropriate  elements  are  identified  by  failure  modes 
which  are  classified  with  respect  to: 

(1)  Location  of  the  modes  in  the  system.  The  nominal  division 
into  subsystems,  components,  and  parts  is  usually  adequate  for  location. 
Some  forms  of  redundancy  and  unusual  inter- component  or  inter- subsystem 
interaction  may  require  further  detail.  This  may  be  provided  by  other 
aspects  of  classification. 

(2)  Relation  of  modes  to  function  as  required  to  evaluate 
interaction  and  redundancy. 

(3)  Relation  of  modes  to  any  design  processes  which  affect 
resistance  to  the  failure.  This  kind  of  information  is  useful  to  the 
program  on  which  it  is  generated  only  when  changes  in  design  are 
required.  It  is  customarily  collected  by  designers  and  rarely  preserved 
for  use  in  predicting  new  designs. 
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3.  RELIABILITY  PREDICTION 


3. 1 Prediction. 

While  reliability  and  its  determination  always  have  connotations 
with  respect  to  the  future,  these  are  usually  clear  from  the  application. 
Therefore  it  is  logical  to  limit  the  word  prediction  to  determination  of 
characteristics  of  equipment  which  is  to  be  different  in  some  way  from 
that  on  which  history  is  available.  This  is  compatible  with  the  implication 
of  prediction  as  the  result  of  inference  from  facts  or  accepted  "laws 
of  nature".  Analysis  of  reliability  of  equipment  identical  to  that  which 
provided  the  data  is  adequately  characterized  as  assessment. 

3. 2 Statistical  Populations. 

Reliability  is  a probability.  It  therefore  represents  a 
fraction  of  a population  or,  in  set  theory,  of  a sample  space.  The 
content  of  the  population  obviously  determines  the  meaning  and  there- 
fore the  validity  of  the  reliability  value.  In  many  applications  of 
probability,  identification  of  population  content  is  simple,  even 
trivial.  For  reliability,  however,  a very  careful  examination  is  often 
required.  This  fact  has  not  been  widely  recognized,  resulting  in  frequent 
invalid  conclusions,  even  among  those  who  know  probability  theory. 
Statisticians  typically  determine  populations  from  component  labels. 

For  assessment  (based  on  tests  of  actual  system  components)  this  is 
sufficient.  But  for  prediction,  new  or  modified  components  must  be 
represented  by  a population  that  is  related  to  the  source  of  the  new  or 
modified  characteristics.  This  requires  the  joint  expertise  of  the 
designer,  who  establishes  these  characteristics,  and  the  statistician, 
who  must  interpret  them  so  as  to  define  a valid  population.  Since  a 
population  is  a probabilistic  concept,  it  is  necessary  for  the  statisti- 
cian to  take  the  initiative  in  correlating  the  two  disciplines.  There- 
fore he  must  understand  the  basic  concepts  of  the  design  well  enough  to 
be  able,  in  consultation  with  the  designer,  to  interpret  experience  in 
terms  of  design  processes  and  characteristics.  He  must  also  understand 
the  significance  and  validity  of  the  scientific  method  as  quoted  from 
Reference  3 in  Section  1.2.  Apparently  the  need  for  this  has  not 
lessened  in  some  25  years  after  being  emphasized  by  Dr.  Fisher. 

3. 3 The  Perspective  of  Prediction. 

Prediction  refers  to  future  events  such  as  aspects  of  equipment 
operation.  To  be  valid  or  accurate  it  must  be  based  on  antecedents  or 
causes,  as  discussed  in  Section  1.1.  The  causes  are  critical  in  deter- 
mining the  "best"  population  to  represent  the  future  events,  as  discussed 
in  Sections  1.3  and  3.2.  One  population  is  better  than  another  if  its 
elements  and  parameters  are  better  known  or  established  in  relation  to 
the  scientific  disciplines  used  in  the  design  process.  These  disciplines 
are  established  by  the  scientific  method,  as  discussed  in  Sections  1.1 
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and  1.2.  Relating  population  elements  or  events  to  causes  requires 
experience  data  in  which  events  are  adequately  classified.  Classification 
is  the  crux  of  the  prediction  process,  partly  because  of  anomalies 
discussed  in  Section  2.3,  but  primarily  because  failure  modes  have  not 
been  identified  in  relation  to  design,  as  discussed  in  (3)  of  Section 
2.4.  The  result  of  disregarding  physical  phenomena  in  statistical 
applications  is  discussed  in  Reference  4,  page  52,  cited  in  Section  1.3. 

Dr.  Feller  concludes  his* illustration  with  " — the  naive  reasoning  as 
such  has  not  been  superseded  by  common  sense,  and  so  it  may  be  useful 
to  have  an  explicit  example  of  how  misleading  mere  goodness  of  fit  can 
be."  The  naivety  in  prediction  is  in  choosing  indirect  population 
determinants  (such  as  labels  or  functional  identifications)  with  little  / 

relation  to  the  event  phenomena. 

3.4  Prediction  Methodology. 

Although  an  optimization  of  statistical  prediction  cannot  be 
strictly  defined,  the  greatest  potential  improvement  lies  in  a better 
understanding  of  the  relationship  of  available  experience  data  to  the 
phenomena  involved  in  the  event  or  events  associated  with  the  prediction. 

This  was  interpreted  statistically  in  terms  of  population  definition  in 
Section  1.3  and  3.2,  and  related  to  application  of  prediction,  as  in 
Section  1.4.  Such  applications  provide  a context  for  the  detailed 
discussion  to  follow.  Examples  of  statistical  analysis  in  the  familiar 
area  of  food  products  will  be  used  to  provide  simple  analogies  to  such 
sciences  as  mechanics  and  electronics,  the  principles  of  which  are  obscure 
to  others  than  specialists.  Some  details  assumed  about  the  examples 
may  be  chosen  to  make  them  simpler  or  more  analogous  although  not 
necessarily  typical  or  realistic. 

3.5  Direct  Population  Determinants. 

Predicting  the  future  yield  per  acre  of  a crop  like  wheat  is 
simple  if  direct  experience  histories  are  available.  "Direct"  is  used 
to  indicate  that  the  experience  covers  specifics  of  the  future  crop, 
such  as  kinds  of  wheat,  location  of  acreage,  etc.  Acceptance  of  such 
characteristics,  both  for  the  problem  and  the  experience,  is  so  routine 
that  it  is  rarely  realized  how  many  aspects  of  their  validity  depend  on 
the  expertise  of  specialists.  Identification  of  subspecies  and  varieties 
of  wheat  grown  in  the  United  States  requires  a competent  horticulturist. 

And  yet  they  differ  in  specific  morphological  characteristics  which  are 
"deterministic"  or  "proved"  as  will  be  described  in  Section  3.9.  But 
when  the  significance  of  the  characteristics  being  identified,  or  the 
nature  and  function  of  the  item,  are  not  so  well  understood  or  so  familiar, 
the  identification  may  be  regarded  as  hypothetical  by  statisticians. 

This  situation  is  typical  in  reliability  analysis  of  electronic  and 
mechanical  equipment.  The  attitude  is  reinforced  by  preoccupation  with 
demonstration  testing,  the  evaluation  of  which  is  deliberately  and  pro- 
perly independent  of  all  information  except  the  test  criteria  and 
results. 
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3. 6 Indirect  Population  Determinants. 


Classifications  in  electronics  and  mechanics  may  appear 
superficially  to  be  analogous  to  those  in  botany.  Components  such  as 
amplifiers  and  motors  may  seem  as  similar  to  others  of  the  same  name 
as  are,  for  example,  different  varieties  of  wheat.  While  this  may  be 
true  for  some  items,  it  can  be  deceptively  wrong  for  others.  In  general 
the  parts  of  similarly-named  components  may  differ  drastically.  The 
reason  is  that  the  names  are  usually  related  to  their  functional 
interrelationship  with  other  components  in  the  context  of  system  func- 
tions. An  example  of  the  effect  of  a name  occurred  during  modification 
of  the  TITAN  missile  to  serve  as  a GEMINI  booster.  A "demodulator"  in 
the  control  system  had  experienced  a number  of  failures  and  was  rede- 
signed. But  a "reliability  prediction"  indicated  no  expected  improve- 
ment. It  developed,  however,  that  the  analyst  ("reliability  engineer" 
or  statistician)  had  not  consulted  the  designer  and  thus  the  "prediction" 
was  not  even  remotely  related  to  the  redesign.  This  particular  demodula- 
tor was  so  named  because  the  demodulation  was  functionally  significant, 
although  employing  a very  small  fraction  of  the  parts.  This  portion  was 
not  redesigned  because  failures  had  not  occurred  there.  Instead  they 
had  occurred  in  an  amplifier  in  the  same  enclosure,  which  was  changed 
from  a "transistor  amplifier"  to  a "magnetic  amplifier".  A prediction 
using  appropriate  data  and  identifications  indicated  substantial  improve- 
ment, later  verified  by  experience. 

3. 7 Choosing  Population  Determinants. 

Direct  or  deterministically  correlated  information  will 
improve  prediction,  even  for  "very  similar  components.  This  will  be 
illustrated  by  use  of  an  example  of  prepared  foods.  Assume  that 
nutritionists  have  developed  "new"  foods  for  some  particular  and 
critical  use  such  as  for  an  expedition  or  exploration.  Assume  that 
there  is  much  nutritional  data  available  on  the  effects  of  foods  on 
health  in  terms  of  maintaining  various  indices  at  least  as  high  as 
accepted  standards.  Statisticians  then  make  an  analysis  to  predict 
the  probability  that  the  newly  "designed"  foods  will  maintain  these 
indices.  Two  kinds  of  data  may  be  used.  One  is  "field  data"  in 
which  the  food  is  identified  by  labels  of  "dishes"  as  found  on  a menu. 

The  other  is  "laboratory  data",  the  results  of  studies  by  nutritionists, 
in  which  the  effects  of  foods  are  evaluated  with  respect  to  ingredients, 
processing,  etc.  of  the  foods.  The  "ingredients"  are  not  those  identified 
by  the  names  found  in  recipes.  Instead  they  are  the  result  of  the  science 
of  nutrition  and  are  based  on  deterministic  identification  of  the  need  for 
and  utilization  of  various  constituents  of  foods  by  the  body.  It  is 
doubtful  if  any  statistician  would  use  the  "field  data"  as  described. 
Everyone  is  familiar  with  proteins,  carbohydrates,  the  multiplicity  of 
vitamins,  etc.,  and  it  would  be  obvious  that  the  names  of  dishes, 
especially  with  modified  or  "new"  recipes,  would  correlate  poorly  with 
nutrition.  But  to  those  who  do  not  understand  electronics,  the  inside 
of  "black  boxes"  is  something  like  magic  and  wizardry  of  the  designer 
must  be  examined  in  the  independent  light  of  the  real  world. 
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Field  data  or  experience  becomes  the  "proof  of  the  pudding",  or  the  real 
world,  and  design  information  is  considered  "hypothetical"  or  "laboratory". 
But  the  determination  of  the  pudding  is  definitely  independent  of  the  eat- 
ing, and  the  greatest  potential  for  improvement  lies  in  influencing  the 
cook  during  its  preparation. 

3. 8 Modified  Assessment  Data. 

Typical  reliability  data  classifies  failure  modes  in  terms 
of  location  in  the  system,  or  in  relation  to  functions,  as  indicated  in 
(1)  and  (2)  of  Section  2.4.  Prediction  of  new  designs  have  been  made 
with  such  data  by  using  certain  "modification  factors"  labeled  "state- 
of-the-art",  "complexity",  etc.,  or  an  attempt  has  been  made  to  identify 
"analogous  equipment",  essentially  based  on  nomenclature.  The  point  of 
view  represented  in  this  simplistic  approach  is  illustrated  by  the  follow- 
ing intended  commendation:  "Prediction  is  based  on  actual  field  data 

rather  than  hypothetical  or  laboratory  data."  Although  field  data  is 
essential  for  assessment,  stresses  are  rarely  instrumented  and  design/ 
strength  related  modes  are  usually  not  identified.  In  contrast, 
"laboratory"  data  results  from  design  testing  in  which  well-instrumented 
stresses,  both  functional  and  environmental,  are  typically  increased  to 
measure  critical  strength  factors  such  as  yield  point,  ultimate  strength, 
etc. 


3. 9 Prediction  Data. 

Occasionally  the  relation  of  a failure  mode  to  the  function  of 
an  item  identifies  the  design  process  affecting  reliability.  This  occurs 
mainly  for  electronic  equipment  and  is  virtually  the  only  data  preserved 
which  is  directly  applicable  to  prediction  of  new  or  modified  designs. 

A recent  study  referenced  in  Section  1.4  (Reference  8,  see  page  23  ) found 
lack  of  such  data  to  be  the  most  critical  present  problem  limiting 
reliability  prediction,  in  particular  for  structures  and  mechanisms. 

Those  who  are  skilled  in  the  disciplines  of  statistics  and  analysis,  and 
are  therefore  able  to  use  the  data,  generally  do  not  appreciate  the 
interrelations  of  their  disciplines  with  those  of  the  physical  sciences. 
Several  widely  accepted  texts  on  applications  of  statistics,  such  as 
those  mentioned  in  Section  1.1,  have  recognized  this.  Reference  2,  page  1 
(see  page  23  herein) , describes  the  disciplinary  interrelations 
explicitly,  beginning  with  the  statement  that  "Natural  and  physical  laws 
are  hypotheses  which  have  been  subjected  to  various  tests  and  have  become 
accepted  or,  as  some  say,  proved."  The  conclusion  is  that  " — statistical 
proof  is  the  basic  form  of  proof  used  in  the  investigation  of  all  sciences." 
A prevalent  and  naive  idea  that  law  indicates  rigorous  certainty  is  then 
addressed,  showing  that  this  is  confusing  statistical  proof  with 
mathematical  proof,  and  that  the  latter  is  " — available  only  in  the 
framework  of  mathematics  itself  but  cannot  be  applied  outside  that  field." 
Thus  a law  is  accepted  when  experience  indicates  that  the  probability 
of  accepting  it  if  false  ("error  of  the  second  kind")  is  small  enough  to 
be  considered  negligible. 
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3.10  Prediction  Analysis. 


Most  design  processes  are  based  primarily  on  scientific  prin- 
ciples which  (Section  3.9)  have  been  established  beyond  reasonable  doubt. 
The  sources  of  unreliability  (with  respect  to  design)  therefore  lie 
elsewhere.  They  may  be  described  as  follows: 

(1)  Variation  in  properties  of  the  materials  or  constituents 
of  components  that  affect  resistance  to  failure.  Related  failures  were 
discussed  in  (2)  and  (6)  of  Section  2.3. 

(2)  Uncertainty  of  stress  determination.  Stresses  corre- 
sponding to  those  described  in  (2)  and  (3)  of  Section  2.3,  may  be 
dominant  in  determining  reliability. 

(3)  Empirical  design  processes.  These  are  relatively  rare 
and  relate  to  phenomena  that  are  beyond  the  "state-of-the-art"  in  that 
they  are  not  "understood"  in  terms  of  scientific  principles.  Though 
rare,  they  may  be  a source  of  serious  design  problems.  A notable 
example  occurred  in  the  Apollo  program,  consisting  of  certain  forms 

of  instability  in  the  large  booster  engines.  For  obvious  reasons 
these  problems  attract  much  attention,  giving  a false  impression  of 
prevalence.  They  do  not  necessarily  result  in  an  unreliable  product, 
as  indicated  by  performance  of  the  Apollo  engines. 

(4)  Distribution  of  stresses.  Except  where  heat  is  critical, 
this  is  easily  determined  for  electronic  equipment.  It  is  more  important 
for  mechanisms  and  is  usually  dominant  for  structures. 

(5)  Empirical  configuration  factors.  A "part"  is  that  which 
may  be  utilized  by  a designer  in  terms  of  its  characteristics  as  an 
entity.  A "component"  is  an  assembly  of  parts,  integrated  to  perform 

a more  complex  function  than  a part.  Typically  component  reliability 
can  be  predicted  by  evaluating  stresses  on  component  parts,  even  when 
the  configuration  introduces  factors  that  are  not  a simple  composite  of 
the  individual  parts  factors.  But  there  are  exceptions  where  the 
configuration  factors  are  not  sufficiently  understood  (as  affecting 
reliability)  and  can  only  be  treated  empirically.  When  the  component  is 
produced,  its  reliability  is  measured  from  test  and  use  data.  Why  not 
preserve  such  data  for  prediction?  It  is  preserved,  but  is  related  to 
component  nomenclature  or  function  and  not  identified  with  design-related 
failure  modes,  as  discussed  in  Section  2.4  under  (3). 

Designers  are  generally  aware  of  these  factors  affecting 
reliability  and  give  them  consideration  within  the  limits  of  general 
techniques.  The  real  potential  for  reliability  improvement  however 
lies  in  the  details  of  multiple  applications  to  highly  complex  systems. 
Techniques  for  resolving  these  are  being  continually  developed  in  the 
design  of  ongoing  programs,  but  the  problem  is  preservation  of  this 
experience  for  future  use.  When  systems  were  relatively  simple, 
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experience  was  preserved  and  easily  located  in  the  literature.  A 
parallel  problem  has  been  preserving  functional  design  techniques,  and 
some  companies  have  attempted  to  solve  this  by  requiring  "official" 
designers  notebooks.  But  there  have  been  no  resources  to  provide 
indexing  for  retrieving  the  new  from  a preponderance  of  routine  techni- 
ques in  the  mass  of  stored  notebooks.  It  is  obvious  that  a program 
cannot  bear  the  cost  of  preserving  experience  for  future  programs. 

Data  banks  are  provided,  however,  within  the  Department  of  Defense  to 
store  and  retrieve  information  for  reliability  analyses.  These  should 
include  prediction  data. 


4.  DEVELOPING  AN  EXPERIENCE  BASE 

4.1  Information  Required. 

As  described  in  Section  3.2,  reliability  prediction  requires 
that  a population,  a statistical  concept,  be  defined  in  terms  of 
characteristics  established  by  the  physical  sciences.  But  these  charac- 
teristics are  those  that  relate  to  failure  modes  in  a probabilistic 
sense  and  are  not  those  that  are  of  primary  interest  to  designers,  whose 
major  concern  is  functional  capability.  In  order  to  utilize  the  expertise 
of  designers,  which  is  essential  to  defining  the  population,  it  is  neces- 
sary for  the  statistician  to  interpret  the  designers  concepts  properly. 

Thus  he  must  understand  the  principles  of  scientific  inference  well 
enough  to  identify  the  determinants  of  reliability.  The  crux  of  this 
identification  for  the  statistician  is  that  "deterministic  relationships, 
which  he  is  inclined  to  distrust,  are  the  essential  identifiers  for  the 
"probabilistic"  emements  of  the  populations  that  determine  reliability. 

4.2  Design  Testing. 

Data  from  design  testing  is  usually  deliberately  ignored  by 
statisticians.  Some  refer  to  it  as  "laboratory  data"  with  the  implica- 
tion that  conditions  are  favorable.  In  fact  the  opposite  is  true,  as 
described  in  Section  3.8.  The  tests  are  often  criticized  as  numerically 
inadequate  to  be  statistically  significant.  This  is  quite  true  if  they 
are  to  be  regarded  as  independent  assessments.  But  typically  these 
tests  are  only  partly  "statistical".  In  addition  they  identify  reliability 
determinants  related  to  known  populations.  The  statistical  aspects  consist 
primarily  of  "safety  margins"  which,  with  identified  strength  distributions, 
provide  reliability  assessment  equivalent  to  much  more  extensive 
"independent"  testing.  Occasionally  designs  are  tested  which  include 
empirical  factors,  as  described  under  (3)  of  Section  3.10.  These  designs, 
as  well  as  some  with  complex  deterministic  phenomena,  may  include  very 
involved  statistical  factors.  If  statisticians  could  correlate  their 
expertise  with  that  of  designers  and  achieve  a rapport,  they  would  not 
only  obtain  data  for  analyses,  but  could  often  make  a contribution  that 
would  improve  reliability  during  the  design  process.  Communication 
between  designers  and  analysts  has  been  stressed  recently  as  part  of 
the  current  emphasis  on  integrated  logistics  support,  and  evidence  of 
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success  is  being  cited.  There  is,  however,  no  evidence  that  this  has 
led  to  recognition  of  design  processes  as  the  direct  source  of  the 
characteristics  of  the  product  being  designed. 

4. 3 Reliability  Testing. 

Most  of  the  testing  that  is  specifically  for  reliability  is 
qualification  or  acceptance  testing.  Instrumentation  is  typically  that 
required  to  establish  the  specified  test  conditions.  For  the  test 
results  to  be  useful  for  valid  prediction,  it  is  necessary  to  determine 
the  stresses  significant  for  all  failure  modes.  Thus  additional 
instrumentation  should  be  provided  as  required  to  measure  them. 

As  explained  in  Section  2.4,  a system  "consists"  of  failure 
modes  with  respect  to  reliability.  These  modes,  when  identified  with 
respect  to  strengths  and  stresses,  represent  those  "elements"  of  a sys- 
tem that  are  most  basic  with  respect  to  reliability  analysis.  Thus 
when  modification  factors  are  derived  to  assess  the  effects  of  various 
operating  conditions  on  reliability,  they  will  be  correlated  with  the 
characteristics  that  are  directly  affected  by  these  conditions.  The 
point  of  view  that  ignores  the  physical  phenomena  probably  arises 
from  the  criteria  for  demonstration  testing,  where  it  is  quite  proper. 
But  it  is  misplaced  when  applied  to  prediction,  where  modification 
factors  must  be  used.  When  these  factors  are  not  associated  with  true 
physical  properties,  they  are  usually  associated  with  names  or  labels. 
Thus  if  competing  designs  used  the  same  component  labels,  the  prediction 
for  each  would  be  the  same. 


5.  DEVELOPMENT  PREDICTION 

Occasionally  during  development  of  a system  a design  proves  to 
be  unacceptable,  resulting  in  a redesign.  Such  an  event  must  be  properly 
identified  in  evaluating  data  for  reliability  prediction  in  order  not 
to  class  it  inadvertently  as  a "failure"  in  that  context  (see  (3), 

Section  2.3).  If  this  experience  were  preserved,  however,  it  would 
provide  a data  base  for  evaluating  and  predicting  future  development 
programs.  The  classification  would  include  characteristics  similar  to 
those  for  reliability  prediction.  Experience  should  reveal  others, 
related  to  the  development  process.  This  information  would  enable 
meaningful  predictions  to  be  made  in  two  important  areas  of  development. 

5. 1 Risk  Analysis. 

A development  program  involves  many  risks.  Among  the  more 
difficult  to  determine  are  those  related  to  design  of  "new"  items. 

Note  that  the  newness  does  not  include  unknown  or  purely  theoretical 
factors,  otherwise  the  program  is  research,  not  engineering  development. 
Thus  it  is  primarily  the  application  that  is  new  and  is  the  source  of 
the  risk.  Typically  only  a small  percentage  of  those  items  that  are 
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nominally  new  are  actually  so.  Unfortunately  analysts  do  not  look 
deeply  enough  to  identify  these  few  items.  Only  in  rare  instances  is 
there  found  in  the  literature  on  prediction  even  the  suggestion  that 
scientific  knowledge  be  utilized.  One  example  has  been  found  in  a 
report  on  mechanical  reliability  prediction  (Reference  11,  Section  4.1, 
page  10)  in  which  the  influences  on  reliability  are  defined  mathemati- 
cally as  "independent  variables".  The  analyst  is  to  determine  the  "best 
set  of  variables".  The  method  for  this  is  described  as  multiple  linear 
regression,  but  then  it  is  pointed  out  that  for  effective  application  of 
this,  "...careful  preliminary  planning  and  engineering  analysis  are 
required."  It  is  further  stated  that  "...the  analyst  must  not  depend 
too  heavily  on  the  computerized  statistical  screening  procedure.  Rather, 
he  must  make  full  use  of  his  scientific  insight,  judgement,  and  experience 
in  selecting  the  best  prediction  equation  from  the  many  mathematical 
possibilities  that  may  appear  equally  good."  Presumably  "engineering 
analysis",  "scientific  insight",  and  "experience",  are  exercised  by 
examining  the  design  of  the  equipment  being  analyzed.  But  why  should  the 
analyst  attempt  to  do  this?  It  is  in  fact  more  difficult  to  deduce  the 
stress-strength  relationships  from  a description  of  the  results  of  a 
design  (intended  to  provide  information  for  fabrication)  than  to  produce 
the  design  itself.  When  a prediction  is  being  made,  the  design  would 
normally  be  just  completed  and  information  readily  available  from  the 
designer.  Otherwise,  the  analyst's  "engineering  analysis"  would  nece- 
ssarily be  based  on  functional  characteristics  and  nomenclature,  which 
obviously  would  not  correlate  as  well  with  failures  as  would  stress- 
strength  relationships. 

5.2  Reliability  Growth. 

Certain  anomalies  exist  in  the  data  used  for  reliability  growth, 
usually  resulting  in  early  estimates  that  are  too  low  and  ultimately  an 
indication  of  more  growth  than  actually  occurs.  Obviously  errors  of 
fabrication  (e.g.,  wiring  errors)  that  are  corrected  by  routine  checking 
are  not  a source  of  true  equipment  failures.  Obvious  errors  are  recognized 
as  such,  but  due  to  inattention  to  failure  phenomena,  many  errors  that 
require  an  understanding  of  the  equipment  are  not  recognized.  If  a 
conservative  "lower  limit"  is  used,  then  "improvement"  due  to  more  data 
should  be  identified  as  information  growth,  not  reliability  growth.  In 
general,  true  growth  should  reflect  changes  in  the  equipment.  If  better 
fabrication  methods  are  developed,  this  is  also  growth,  but  the  source 
should  be  identified.  The  principle  difference  between  reliability 
growth  and  risk  as  described  in  Section  5.1,  is  that  the  planned  effort 
allowed  for  corrective  action  is  not  considered  a "risk".  Thus  a more 
accurate  prediction  of  reliability  will  reduce  both  the  risk  and  the 
necessary  growth. 


* ARINC  Research  Corporation,  Development  of  a Reliability  Prediction 
Procedure  for  Shipboard  Mechanical  Equipments  - Phase  I,  ARINC 
Research  Publication  No.  933-01-2-1079,  ARINC  Research  Corporation, 
Anapolis,  MD,  November  1970. 


20 


6.  CONCLUSIONS  AND  RECOMMENDATIONS 


In  Section  3.10  under  (5),  reference  was  made  to  prediction 
of  component  reliability  from  data  on  parts.  Techniques  for  this  were 
presented  in  MIL-HDBK-217A  (cited  in  Section  2.1).  Simple  design 
procedures  for  parts  selection  were  included,  for  example  the  derivation 
of  semi-conductor  junction  temperature  (under  7.4.3),  usually  the 
primary  reliability  factor  for  these  devices.  Parts  selection  techniques 
have  been  expanded  and  updated  in  the  RADC  Reliability  Notebooks. 
(Reference  12) . These  techniques  have  been  used  mainly  in  the  aerospace 
industry,  and  lack  of  general  acceptance  is  indicated  in  the  literature 
on  reliability  prediction  where  use  of  data  on  parts,  if  mentioned,  is 
usually  considered  unrealistic.  But  this  is  only  one  factor  in  such 
prediction,  and  is  usually  not  the  critical  factor.  Instead  the 
potential  unreliability  of  "new"  designs  lies  in  design  configuration 
factors  discussed  in  Section  3.10  under  (5).  The  one  recognition  of 
this  found  in  the  literature  (See  Section  5.1)  failed  to  recognize 
designers  as  the  logical  source  of  information  on  which  to  base  reli- 
ability determinants  of  the  designs  they  produce. 

In  order  to  further  develop  a valid  approach  to  prediction 
of  reliability,  and  to  extend  it  to  other  areas,  three  things  are 
necessary:  (1)  analysts  must  learn  enough  about  the  scientific  method 
of  engineering  design  to  ask  designers  pertinent  questions  and  to  use  the 
answers  properly;  (2)  the  information  obtained  must  be  used  to  collect 
and  preserve  histories  of  events  and  to  relate  them  to  design  procedures; 
(3)  it  must  be  further  used  to  develop  methods  of  analyzing  new  designs 
and  to  make  predictions  based  on  the  data. 

6. 1 Correlating  Analysis  with  Design. 

It  should  be  obvious  that  any  analysis  which  attempts  to 
predict  some  characteristic  expected  to  result  from  a design  should 
consider  all  available  evidence  relating  to  the  effect  of  the  design 
on  that  characteristic.  It  should  further  be  obvious  that  the  prime 
design  processes  to  be  considered  are  those  having  the  explicit  purpose 
of  determining  the  characteristic.  In  order  to  do  this,  analysts  must 
understand  and  accept  the  statistical  significance  of  natural  and 
physical  laws  as  cited  in  Section  3.9  from  Reference  2,  and  be  able  to 
differentiate  these  from  factors  in  the  design  process  that  are  subject 
to  significant  statistical  variation.  The  primary  obstacle  to  overcome 
is  apparently  a misapplication  of  a principle  of  "independence",  which 
properly  applies  to  testing  a product  after  it  is  produced. 


12RADC  Reliability  Notebooks,  Vol  I (AD  845304)  and  Vol  II  (AD  821640). 
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6. 2 Developing  Prediction  Methodology. 


A "good"  prediction  technique  must  identify  the  more  significant 
"causes"  of  the  "effect"  that  is  being  predicted.  When  considering 
reliability,  the  phenomenon  of  failure  can  always  be  interpreted  in  terms 
of  a "stress"  exceeding  a "strength".  The  techniques  that  have  been  used 
have  usually  identified  conventional  strengths,  as  those  of  parts  listed 
in  MIL-HDBK-217A.  But  in  a more  general  sense,  a strength  may  be  any 
property  or  characteristic  of  a part  or  an  interrelated  assembly  of  parts 
that  has  an  effect  on  successful  operation  of  a system.  The  word 
"balance"  may  represent  such  a characteristic  in  either  electronic  or 
mechanical  design.  A complex  of  factors  may  contribute  to  an  essential 
characteristic  of  balance  and  thus  determine  success  or  failure.  A true 
advance  in  the  state  of  the  art  of  reliability  prediction  must  correlate 
failures  with  such  failure  determinants  of  the  design  process  rather  than 
with  functionally  oriented  nomenclature  of  components. 

6.3  Preserving  Experience  History. 

The  basis  of  any  prediction  technique  must  be  a properly 
correlated  experience  history.  The  correlation  must  be  with  whatever 
identifies,  defines,  or  determines  that  which  is  being  predicted.  Thus 
a data  base  for  predicting  the  expected  reliability  of  equipment  from  a 
design,  must  relate  failures  to  design  processes  that  affect  the  proba- 
bility of  failure.  This  was  done  in  MIL-HDBK-217A  for  selection  of 
parts,  but  should  be  extended  to  other  factors  of  the  design  process. 

Such  factors  often  dominate  determination  of  reliability  in  mechanical 
design.  The  task  of  developing  a data  base  must  precede  analytical 
methods  in  order  that  the  latter  be  application  oriented.  Reference  8 
(see  page  23)  states  in  Section  1,  page  1:  "The  first  step  in  the 

application  of  mathematics  to  a physical  problem  is  the  construction  of 
a mathematical  model  which  embodies  the  main  observational  and  theoreti- 
cal features  of  the  phenomenon."  Thus  it  is  necessary  to  obtain 
experience  data  from  which  the  phenomena  may  be  identified  before  they 
may  be  embodied  in  the  models.  Such  identification  requires  that 
expertise  be  provided  by  experienced  designers  in  all  fields  related  to 
the  systems  of  interest.  Thus  the  initial  task  in  developing  a predic- 
tion capability  should  be  carried  out  by  a team  of  specialists  in  design, 
instrumentation,  and  data  collection.  Only  elementary  aspects  of 
statistics  will  be  involved. 
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APPENDIX 


ENGINEERING  RELIABILITY  PROGRAM 


1.  OBJECTIVE 

When  products  were  relatively  simple  and  design  latitudes 
generous, • adequate  reliability  was  achieved  by  careful  conservative 
design.  Except  for  such  design  parameters  as  these  associated  with 
safety  factors,  reliability  was  mainly  good  intentions  of  a general 
nature,  rarely  quantified.  Increasing  complexity  and  constraint  have 
revealed  the  limitations  of  the  older  approach,  and  the  advantage  of 
a rigorous  discipline  using  statistical  methods  has  been  demonstrated. 

It  is  the  objective  of  a reliability  program  to  produce  a statistical 
quantification  of  design,  test,  and  use  information,  to  determine  the 
interrelation  among  these  by  appropriate  analyses,  and  to  use  the  results 
as  a basis  for  design,  component,  and  integration  control  in  order  to 
achieve  a high  probability  of  correlation  of  the  final  product  with 
the  design  intent  and  the  test  and  use  indices. 


2.  SCOPE 

The  reliability  effort  is  accomplished  by  specific  tasks 
(listed  later  under  the  heading  of  "Method")  which  may  be  divided  into 
three  general  areas  as  follows: 

(1)  Reliability  of  design.  Tasks  8 and  9. 

(2)  Selection  and  control  of  components,  Tasks  2,  3,  4,  5,  and  6. 

(3)  Appraisal  of  reliability  achievement  and  potential.  Tasks  1 

and  7. 


3.  METHOD 

The  following  tasks  are  required  to  carry  out  the  reliability 
program  and  are  listed  under  three  categories  which  identify  the  major 
method  involved. 

a.  Reliability  Control  and  Documentation 

Task  1,  Program  Review.  The  proposal  would  describe  the 
establishment  of  data  control  procedures  for  information  on:  component 

reliability  experience;  design  analysis  and  reviews;  time/cycle  and 
storage  sensitive  components.  The  purpose  is  to  provide  information  to 
assess  the  reliability  status  of  the  product  and  to  contribute  the 
reliability  portion  of  program  reports. 
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Task  2,  Review  of  Test  Program  and  Procedures.  This  effort 
will  ensure  that  the  testing  is  compatible  with  reliability  requirements 
and  that  data  is  provided  for  assessment. 

Task  3,  Failure  Analysis  and  Corrective  Action.  Failure 
analysis  will  provide  the  basis  for  corrective  action  appropriate  to 
achievement  of  adequate  reliability.  The  proposal  should  describe 
the  means  of  executing  this  operation. 

b.  Vendor  Selection  and  Component  Control 

Task  4,  Review  and  Evaluation  of  Vendor  Reliability 
Programs  and  Activities.  This  task  determines  the  adequacy  of  vendor 
reliability  programs  and  provides  an  input  to  vendor  selection. 

Task  5,  Identification  and  Control  of  Time/Cycle  and  Storage 
Sensitive  Components.  The  proposal  would  show  how  this  task  ensures 
that  components  are  not  degraded  prior  to  actual  use. 

Task  6,  Component  Selection  and  Control.  The  objective  of 
this  task  is  selection  of  components  with  demonstrated  reliability  or 
with  potential  reliability  to  be  achieved  through  diligent  application 
of  testing,  reliability,  and  quality  control  programs.  The  proposal 
should  show  how  these  programs  are  integrated  and  coordinated  and  how 
component  control  provides  a close  statistical  correlation  between  the 
tested  samples  and  the  components  used  in  the  final  product. 

c.  Reliability  Analysis 

Task  7,  Reliability  Prediction  and  Assessment.  The 
proposal  would  describe  how  an  initial  prediction  and  allocation  is 
made  and  revised  as  changes  are  made  in  preliminary  designs.  Later 
in  the  program,  data  from  testing  actual  hardware  will  provide  the 
basis  of  assessments. 

Task  8,  Reliability  Design  Analysis.  The  proposal  should 
show  how  this  effort  which  is  concurrent  with  designs,  results  in 
reliability  being  "designed  into"  the  subsystems.  The  analyses  include 
evaluation  of  reliability  problem  areas,  use  of  redundancy,  and  trade- 
offs involving  reliability.  Participating  in  design  reviews  is  part 
of  this  task. 

Task  9,  Review  of  Proposed  Design  Changes.  Continuous 
monitoring  of  subsystem  design  provides  the  background  for  review  of 
changes  for  effect  on  reliability. 
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4.  IMPLEMENTATION 


Information  and  techniques  are  generally  available  for  carrying 
out  most  of  the  tasks  described.  The  state-of-the-art  for  reliability 
prediction  and  design  analysis  (Tasks  7 and  8)  for  structures  and 
mechanisms  is,  however,  seriously  lagging,  and  for  electronics  is  out  of 
date.  Improvement  is  needed  in  three  areas,  failure  mode  analysis, 
reliability  design  analysis,  and  preservation  of  the  resulting  information 
or  data.  These  will  be  discussed  in  some  detail. 

(1)  Failure  mode  analysis.  In  order  to  use  failure  data  as  a 
basis  of  predicting  the  reliability  of  an  item  that  has  not  yet  been 
fabricated,  it  is  essential  that  the  failure  modes  be  determined  in  terms 
of  identified  strengths  that  have  been  exceeded  by  identified  stresses. 

Only  then  is  it  applicable  to  the  design  constants  and  specifications 
that  describe  a proposed  item. 

(2)  Reliability  design  analysis.  The  strengths  significant 
to  failure  modes  must  be  related  to  design  "outputs"  or  constants  which 
establish  them,  and  the  constants  in  turn  related  to  the  design  disciplines 
and  techniques  by  means  of  which  they  are  derived.  This  includes  a 
statistical  analysis  resulting  in  probability  of  identified  stresses 
being  exceeded  by  identified  strengths.  In  some  cases  failure  mode  and 
reliability  design  analyses  will  not  establish  the  required  identifications. 
If  these  failures  can  be  assigned  some  kind  of  modal  classification,  then 

a statistical  correlation  may  be  sought  with  known  design  constants.  If 
such  is  found  the  mode  is  thus  identified  for  use  in  prediction. 

Correlations  so  discovered  provide  potential  clues  for  designers  to  look 
for  a means  to  control  strengths  related  to  these  failure  modes.  Finally 
a failure  analysis  may  not  reveal  sufficient  information  for  any  modal 
classification.  Such  failures,  sometimes  called  "random",  can  be  identi- 
fied only  with  respect  to  the  "type"  of  item  to  which  they  are  applicable 
and  used  for  prediction  in  this  manner.  Thus  three  "kinds"  of  failures 
are  recognized  which  may  be  labeled,  in  order  of  discussion,  as  intrinsic, 
empirical,  and  random. 

(3)  Data  system.  The  information  generated  by  the  analyses 
described  must  be  preserved  along  with  the  usual  "assessment"  data.  A 
design  engineer  might  point  out  that  information  of  this  kind  is 
generated  by  "design"  testing.  Such  tests  are  often  made  on  some 

parts  of  an  item  with  simulated  functional  and/or  environmental  stresses. 
Typically  very  extensive  instrumentation  is  used  and  stresses  are  usually 
increased  to  determine  points  of  failure.  Results  of  such  testing 
should  be  included  in  the  data  system.  Although  it  is  not  feasible  to 
instrument  reliability  testing  as  extensively  as  design  testing,  it  should 
be  done  to  the  extent  practicable  in  order  to  identify  more  "intrinsic" 
failures. 


In  summary,  reliability  testing  should  be  more  like  design 
testing  with  complete  records  preserved,  failures  should  be  analyzed 
to  identify  the  physical  phenomena  involved,  and  the  phenomena  should 
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be  related  to  the  design  processes  and  outputs.  While  these  relations 
are  probabilistic,  in  the  form  of  distributions,  it  is  essential  that  the 
form  and  parameters  of  the  distributions  be  identified  as  closely  as 
possible  with  the  deterministic  physical  relationships  that  form  the 
basis  of  the  design  disciplines  being  used. 
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